1. Field of the Invention
The present invention relates generally to computer security, and more particularly, to controlled testing of a potentially malicious program.
2. Description of the Related Art
The deployment and growth of large computer networks such as the Internet has facilitated the proliferation of malicious software programs known as malware. Malware programs are designed to infiltrate a target host system without the owner's permission in order to exploit the security weaknesses of the system and network. Malware is typically used to steal information stored in the target host systems, such as account numbers, passwords, social security numbers, credit card numbers, etc.
Malware has become the preferred mode of operation for organized crime in the Internet, as malware can be spread with ease via email attachments, web downloads and file transfers. Malware carries code that damages host systems, creates backdoors to networks, allows attackers free entry into the network, redirects search engine results to paid advertisements, creates denial-of-service attacks and takes control of infected host systems.
One form of protection used to protect a target host computer from malware encompasses simulating suspicious programs at the target host computer. A simulation engine creates a simulated environment at the target host system and executes the suspicious program in order to determine whether the program is benign or harmful. The simulated environment is created with characteristics typical of a target host system environment. The simulation engine runs on the target host system in order to have access to the target host system information. Incorporating the target host system information into the simulated environment results in a simulated environment that is very similar to the real target host system environment. However, there are cases where this solution might be impractical or undesirable. For example, when the target host system is a system with limited processing power, performing the simulation locally would be impractical. In addition, there are situations where the analysis needs to be performed at a remote system and not at the target host system, such as when an anti-malware engine is deployed on a network sensor.
Moving the simulation to a remote system is likewise problematic. Savvy malware writers have developed sophisticated malware that analyzes the details of the environment to ascertain whether the malware is being run in a simulated environment or in the target host system. For example, some sophisticated malware determines an arbitrary property of the target host system and makes the execution of malicious actions dependent on the existence of that property. As such, when a simulated environment does not have the specific arbitrary property, the malware determines that the running environment is not the target host environment but instead a simulated environment. Accordingly, the malware will not perform any malicious actions during the simulation in order to avoid detection. The malware may then pose as a benign program that can be safely executed at the target host system. Since no malicious actions were performed, the simulation erroneously establishes that the suspected program is not malware.
Another solution to this problem collects the most common details and properties of a number of target host systems and then creates an average environment for simulation using those details and properties. However, this approach leaves inevitable holes in the security of the network, given that all the details and properties would not be accounted for. For example, let us assume that some target host systems are running one operating system and other target host systems are running a different operating system. The average simulated environment would be set up to run only one of the two operating systems. This solution may protect target host systems running the same operating system as the average simulated environment, but will leave the other target host systems vulnerable.
Another solution to this problem involves enhancing the simulation engine to branch into multiple simulation paths each time the suspicious program queries the environment for specific details. The simulation engine can be configured to branch into a different path for each possible answer. However, this solution requires the simulation engine to perform a large number of computations that increase as the number of queries and possible answers increases. As such, this solution is neither practical nor feasible in view of the computing resources needed to accomplish the myriad calculations.
Therefore, what is needed is a computer system and method for controlling testing of a potentially malicious program.